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I.  INTRODUCTION 


A.  NEED  FOR  THIS  STUDY 

In  major  weapon  system  acquisition,  the  Department  of  Defense  typically  issues 
a  large  number  of  change  orders  to  modify  an  existing  contract.  In  each  case,  the 
contractor  prepares  a  proposal  of  contract  price  adjustment  for  the  requested  changes. 
A  lead  ship  or  aircraft  production  contract  may  generate  over  10,000  such  proposals. 

Acquisition  regulations  require  the  government  to  analyze  each  proposal  and 
negotiate  with  the  contractor  for  a  fair  price  adjustment.  However,  these  requirements 
produce  more  work  than  the  existing  number  of  government  auditors  can  undertake. 
Generally,  as  backlogs  develop,  thoroughness  is  reduced  for  expediency,  and  errors  slip 
by.  However,  work  in  progress  usually  goes  on,  but  contractors  are  not  paid  until  after 
the  proposal  is  processed.  This  delay  in  funding  raises  the  capital  cost  to  the 
contractor  for  performing  work  for  the  government. 

It  is  in  the  best  interests  of  both  the  government  and  the  contractors  to  raise  the 
accuracy  and  speed  and  reduce  the  cost  of  auditing  such  proposals  of  price  adjustment. 
If  it  were  allowed  by  the  acquisition  process,  analyzing  and  negotiating  a  sample  of 
proposals  selected  with  an  effective  sampling  technique  could  have  these  effects. 

The  reason  for  auditing  the  proposal  population  is  to  ensure  the  proposals  are 
realistic  and  fair.  During  an  audit,  a  proposal  will  be  found  to  be  understated,  that  is 
the  Government  feels  that  the  proposed  cost  of  a  change  to  the  contract  is  less  than 
the  actual  cost  the  contractor  will  incur,  or  overstated,  or  correct.  Any  overstatement 
or  understatement  is  considered  an  error  in  the  proposal  population.  The  ideal 
sampling  technique  would  choose  a  sample  from  the  population  of  proposals  which, 
when  audited,  would  always  give  an  estimated  value  for  the  population  as  a  whole 
which  was  exactly  correct,  no  matter  what  the  degree  or  distribution  of  errors  in  the 
proposal  population.  Such  a  technique  does  not  exist. 

Assuming,  then,  that  there  will  be  some  degree  of  error  in  the  prediction  of  the 
true  value  of  the  entire  proposal  population  whenever  a  sampling  technique  is  used;  the 
behavior  of  the  degree  of  error  must  be  predictable  and  exhibit  certain  qualities  in 
order  for  the  sampling  technique  to  be  considered  appropriate  for  the  purpose 
described  above. 
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Specifically,  the  degree  of  error  should  not  be  easily  altered  by  the  error 
arrangement,  that  is  the  distribution,  size,  frequency,  or  type  (overstatement  or 
understatement)  of  errors  found  in  the  population,  as  long  as  the  total  error  is  the 
same.  A  sampling  procedure  which  shows  little  response  to  changes  in  the  error 
arrangement  will  be  said  to  show  consistency.  If  certain  patterns  of  errors  caused  the 
entire  population  to  be  evaluated  as  understated  then  the  government  would  pay  more 
than  a  fair  price  for  the  changes  described  by  the  population  of  proposals  which 
contained  those  errors.  This,  in  turn,  would  tend  to  encourage  fraud,  since  a 
contractor  could  carefully  seed  his  proposal  population  with  deliberate  errors  of  the 
appropriate  size,  type,  and  distribution  and  thereby  be  awarded  a  larger  payment  from 
the  government. 

Therefore,  the  desired  method  will  not  necessarily  be  the  one  which  results  in  the 
most  accurate,  average  estimate  for  various  error  arrangements  in  the  proposal 
population.  It  will  instead  be  the  method  which  responds  least  to  variations  in  the 
error  arrangement. 

B.  METHODOLOGY 

In  this  paper  the  Dollar  Unit  Sampling  Method  and  the  Basket  Method  will  be 
used  to  draw  samples  from  populations  for  evaluation.  The  populations  were  used 
earlier  in  a  joint  study  of  the  American  Institute  of  Certified  Public  Accountants  and 
the  American  Statistical  Association.  Each  population  consists  of  two  columns  of 
values  which  represent  the  proposed,  or  book  value  of  a  contract  change  and  the 
audited  or  true  value  of  that  change.  The  populations  are  rigged  with  either  random  or 
planned  errors.  Samples  drawn  by  the  two  methods  from  each  population  will  be 
evaluated  and  compared  to  determine  which  method  gives  a  better  estimate  of  the 
whole  population  according  to  the  goals  described  above.  Both  the  error  rigging  and 
evaluation  steps  are  explained  further  in  the  description  of  the  simulation. 

The  amount  of  work  associated  with  auditing  is  more  closely  related  to  the 
number  of  items  being  audited  than  to  the  total  dollar  value  of  all  the  items  being 
audited.  Therefore,  the  sampling  rules  of  the  two  methods  will  be  adjusted  so  that  they 
will  draw  samples  with  the  same  number  of  proposals  from  each  population.  The 
results  of  this  study  will  then  indicate  which  method  yields  the  more  desirable 
prediction,  that  is,  the  greater  benefit,  while  holding  the  cost  of  the  audit  constant. 


7 


II.  DOLLAR  UNIT  SAMPLING 


A.  HISTORY 

Dollar  Unit  Sampling  is  a  technique  which  is  strongly  based  on  statistical  theory. 
It  determines  the  possible  error  in  a  sample  based  on  a  Poisson  model  of  the 
population. 

The  roots  of  the  Dollar  Unit  Sampling  method  can  be  traced  to  the  1960's  when 
van  Heerden  developed  the  "guilder  number"  method  [Ref.  1).  About  the  same  time, 
K.  W.  Stringer  developed  a  similar  model  which  he  called  Cumulative  Monetary 
Amounts  (CMA)  Sampling  [Ref.  2].  Dr.  Albert  Teitlebaum  then  developed  a  model 
similar  to  Cumulative  Monetary  Amounts  Sampling  which  he  called  Dollar  Unit 
Sampling  (DUS)  [Refs.  3, 4, 5, 6].  In  December  of  1973  he  presented  the  first  attempt  at 
a  comprehensive  proof  for  Dollar  Unit  Sampling,  which  led  to  rising  prominence  of  the 
Dollar  Unit  Sampling  method  [Refs.  3,7). 

Dollar  Unit  Sampling  is  also  known  as  Monetary  Sampling  (MS),  Combined 
Attribute  Variable  Sampling  (CAV),  Monetary  Unit  Sampling  Technique  (MUST), 
Sampling  Proportionate  to  size  (SPS),  and  Probability  Proportional  to  Size  (PPS) 
[Ref.  8], 

The  CMA  procedure  is  the  most  widely  used  version  of  DUS,  but  it  has  the 
disadvantage  of  overestimating  the  possible  overstatement  [Refs.  7,8,9].  While  this  is  a 
conservative  error,  from  the  government's  viewpoint,  it  is  still  an  error,  and  contractors 
will  be  unlikly  to  bind  themselves  to  a  method  which  will  routinely  overcorrect  against 
them. 

B.  DESCRIPTION  OF  DOLLAR  UNIT  SAMPLING 

There  are  three  steps  to  using  the  Dollar  Unit  Sampling  method:  calculating  the 
sample  size,  choosing  the  sample,  and  calculating  the  error.  These  steps  will  be  briefly 
discussed  below. 

1.  Calculating  Sample  Size 

There  are  four  commonly  used  methods  for  the  determination  of  the  sample 
size.  All  require  the  total  book  value  of  the  contract  changes,  the  reliability  factor  for 
a  given  level  of  confidence,  and  the  amount  of  error  which  can  be  tolerated  for  a  given 
level  of  confidence.  The  better  the  estimated  amount  of  error  in  the  sample  is  known, 
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the  more  accurate  the  required  sample  size  will  be.  In  practice,  tables  are  used  which 
specify  the  sample  size  based  certain  arguments  for  book  value,  level  of  confidence,  and 
estimated  error  rate. 

In  this  study;  however,  the  process  of  selecting  a  sample  size  is  not  required  to 
be  followed.  This  is  because  in  actual  applications  where  the  true  value  of  the 
population  is  not  known  the  only  way  to  be  reasonably  certain  that  one's  results  are 
valid  is  by  complying  with  rules  which  will  tie  the  audit  to  statistical  theory.  The 
sampling  rules  for  Dollar  Unit  Sampling  are  intended  to  do  just  that,  so  that  the 
auditor  who  follows  the  sampling  procedures  will  be  able  to  determine  the  extent  of  the 
audit  required  to  achieve  the  desired  certainty. 

In  this  study  the  true  values  of  the  proposals  are  known,  as  are  the  size  and 
distribution  of  the  errors,  and  the  Dollar  Unit  Sampling  method  is  not  being  compared 
to  its  theoretical  limits,  but  to  a  second  method  to  determine  which  of  the  two  gives  a 
more  desireable  result  in  a  certain  case  for  the  same  amount  of  work.  Therefore,  the 
size  of  the  sample  must  be  based  on  a  number  of  proposals  rather  than  a  number  of 
dollars.  This  being  the  case,  one  tenth  of  the  total  number  of  proposals  in  the 
population  was  arbitrarily  chosen  as  the  size. 

2.  Drawing  the  Sample 

The  heart  of  Dollar  Unit  Sampling  is  the  choosing  of  the  sample.  Rather  than 
a  random  technique,  which  might  select  a  certain  number  of  contracts  to  audit,  the 
sample  size  calculated  in  the  previous  step  refers  to  a  number  of  single  dollar  units. 
Single  dollar  units  are  merely  dollars.  Thus,  a  proposal  which  is  estimated  at  one 
hundred  dollars  would  contain  one  hundred  single  dollar  units.  Each  of  these  dollars 
has  the  same  probability  of  being  selected  as  every  other  dollar  in  the  entire  proposal 
population.  When  a  dollar  unit  is  selected,  the  entire  proposal  in  which  it  resides  is 
audited.  Since  the  single  dollar  units  are  each  given  the  same  chance  of  being  selected, 
a  proposal  with  a  larger  dollar  value  than  another  has  a  larger  chance  of  being  selected 
for  the  audit. 

To  select  the  dollars,  and  thereby  the  proposals,  to  be  audited,  the  proposals 
are  stacked  and  then  added  to  form  a  series  of  partial  sums.  The  proposals  which  lie 
between  the  partial  sums  which  bracket  the  random  numbers  generated  by  the  sizing 
calculations  are  selected  for  auditing.  A  simple  algorithm  for  accomplishing  this  is  to 
subtract  the  proposal  book  values  from  the  random  number  until  the  difference  is 
negative  and  audit  the  last  proposal  subtracted.  The  step  by  step  process  is  as  follows: 

1.  Arrange  the  proposals  in  a  sequence  and  find  the  sum  of  their  book  values. 
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2.  Calculate  the  Average  Sample  Interval  (AS1)  using  the  equation: 

ASI  =  B  /  n 

where 

B  =  sum  of  book  values  and  ... 

n  =  number  of  dollars  to  be  chosen  as  determined  by  the  sample  size  calculation 

3.  Select  a  random  start  number  RS  such  that  0  <  RS  <  ASI. 

4.  Subtract  the  proposal  book  values,  in  the  sequence  determined  in  step  one, 
from  RS  until  the  difference  is  negative. 

5.  Audit  the  proposal  which  makes  the  running  difference  negative,  add  the  ASI  to 
the  negative  difference,  and  repeat  from  step  4. 

If  a  proposal  has  a  book  value  greater  than  the  ASI  it  will  be  chosen.  A 
proposal  is  only  allowed  to  be  chosen  once,  no  matter  how  large  it  is,  so  the  number  of 
dollar  units  calculated  when  determining  the  sample  size  will  be  greater  than  number  of 
proposals  selected  for  audit  if  any  proposals  have  a  book  value  greater  than  the  ASI. 
The  development  of  digital  computers  has  greatly  eased  the  execution  of  this  phase. 

3.  The  Peach  Problem 

Before  discussing  the  evaluation  phase,  it  is  useful  to  briefly  illustrate  and 
develop  the  Dollar  Unit  Sampling  technique  by  examining  the  Teach  Problem'  [Ref.  8]. 
In  the  Peach  Problem  one  must  determine  the  portion  of  peaches  in  a  lot  of  one 
million  which  is  unusable.  To  determine  the  size  of  the  portion  one  will  draw  a  sample 
and  use  information  on  both  the  number  of  rotten  peaches  and  the  extent  of  rottenness 
found. 

One  hundred  sample  peaches  are  randomly  drawn  and  examined.  If  there  are 
no  defective  peaches,  there  is  a  5%  risk  that  the  percentage  of  peaches  defective 
exceeds  3%.  These  numbers  are  based  on  the  Poisson  distribution  which  is  chosen 
because  it  is  a  good  estimater  of  the  binomial  case  being  examined.  The  binomial  is 
that  a  peach  is  either  rotten  to  some  detectable  degree  or  it  is  not.  Similarly,  a 
proposal  is  either  overstated  or  understated  to  some  significant  degree  or  it  is  accurate. 
It  should  also  be  noted  that  the  terms  mentioned  in  the  sample  size  calculation  now 
take  on  values.  The  total  book  value  is  one  million.  The  a  risk,  or  risk  of  accepting 
an  unacceptable  lot,  is  5%,  which  means  the  confidence  is  95%.  The  reliability  factor 
for  the  95%  confidence  level  is  3%,  meaning  that  at  most,  30,000  peaches  could  be 
rotten  to  some  degree  and  the  population  would  still  be  acceptable. 

If  there  had  been  one  defective  peach,  there  would  have  been  a  5%  a  risk  that 
the  percentage  of  defective  peaches  exceeded  4.75%.  Similarly,  if  two  defective  peaches 
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were  found,  there  would  have  been  a  5%  a  risk  that  the  number  of  defective  peaches 
will  exceed  6.31%.  These  values  also  come  from  the  Poisson  distribution. 

What  is  not  considered  yet  is  the  fraction  of  the  defective  sample  peaches 
which  is  rotten.  This  is  important  if  one  can  use  a  partial  peach  or  partial  dollar. 

To  account  for  this  in  the  case  with  two  defective  peaches  in  the  sample,  the 
two  defective  peaches  are  examined.  Assume  the  two  rotten  peaches  were  8%  and  4% 
rotten.  Then  in  95%  of  the  cases: 

1.  The  number  of  totally  rotten  peaches  will  be  less  than  3%. 

2.  The  number  of  8%  rotten  peaches  will  not  exceed  an  additional  1.75%.  (4.75% 


-  3%) 


3.  The  number  of  4%  rotten  peaches  will  not  exceed  1.56%.  (6.31%  -  4.75%) 
Then  the  total  portion  of  the  population  which  is  rotten  will  be  less  than: 


3.00%  x  100% 
1.75%  x  8% 
1.56%  x  4% 


=  3.00% 
=  0.14% 
=  0.06% 


4.62% 


This  is  a  better  estimate  of  the  total  amount  of  rotten  peaches  in  the  lot  than 
6.31%  because  it  corrects  for  the  usable  part  of  the  partially  rotten  or  'tainted' 
peaches.  It  should  be  noticed  that  the  additional  corrections  are  arranged  in  order  of 
decreasing  percent  tainting.  This  gives  the  most  conservative  result  since  the  smaller 
percent  taintings  are  applied  to  the  smaller  percent  defect  increments. 

The  analogy  applies  directly  to  the  proposals  problem.  The  peaches  are 
dollars,  each  one  essentially  the  same  size.  The  degree  of  rottenness  is  the  degree  of 
error  in  the  entire  proposal  in  which  the  sampled  dollar  is  found.  Thus,  if  a  chosen 
dollar  is  in  a  proposal  which,  when  audited,  reveals  an  8%  overstatement  error,  each 
dollar  in  that  proposal,  and  in  particular,  the  dollar  which  was  chosen  to  be  sampled,  is 
considered  8%  wasted. 

4.  Calculation  of  the  Result 

If  there  are  no  errors  in  the  sample,  no  calculation  is  required  and  the  amount 
of  error  for  the  planned  level  of  confidence  is  the  planned,  tolerable  amount  of  error. 

Similarly,  if  the  sample  size  was  determined  using  the  P  Risk  Only  tables,  and 
the  sum  of  the  taintings  in  the  sample  is  less  than  that  planned  when  the  sample  size 
was  determined,  then  the  resulting  tolerable  amount  of  error  is  less  than  that  planned 
and  no  calculation  is  required. 
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If  the  sample  size  was  determined  using  the  a  and  P  risk  tables,  and  the  sum 
of  the  taintings  is  less  than  the  K-factor,1  then  the  actual  error  is  less  than  the  planned, 
tolerable  error  limit;  three  percent  in  the  previous  example. 

If  some  other  result  is  reached  then  some  calculation  will  be  necessary.  Each 
erroneous  proposal  is  listed  in  order  of  decreasing  tainting.  The  taintings  are  then 
multiplied  by  the  incremental  percent  defect.  The  sum  of  the  taintings  and  tainting 
products  are  then  used  to  compute  the  net  upper  error  limits. 

If  any  errors  are  found  materiality  must  be  redefined  since  the  sample  size 
anticipates  that  the  sample  will  contain  no  errors.  If  the  errors  found  are  few  or  small, 
the  new  materiality  might  be  close  enough  to  the  original  goal  to  allow  acceptance  of 
the  population. 


^he  K-factor  is  one  of  the  two  values  read  from  the  tables  used  to  determine  the 
sample  size,  the  other  being  the  sample  size.  The  K-factor  is  significant  because  it  is 
compared  to  the  sum  of  the  tainting  in  the  sample  to  determine  if  the  population  can 
be  accepted  without  exceeding  the  p  risk  (£]  taintings  ^  K-factor)  or  rejected  without 
exceeding  the  a  risk.  (£  taintings  >  K-factor) 
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III.  THE  BASKET  METHOD 


A.  HISTORY 

The  Basket  Method  was  developed  by  Dr.  K.  T.  Wallenius  under  the  sponsorship 
of  the  Office  of  Naval  Research  and  the  Naval  Material  Command.  It  is  a  recently 
created  technique  which  was  formed  in  direct  response  to  the  problems  of  auditing  and 
negotiating  numerous  contract  proposals  with  sole  source  contractors  [Ref.  10]. 

B.  DESCRIPTION 

The  Basket  Method  derives  its  name  from  its  process  of  sorting  a  population  into 
a  number  of  baskets,  one  of  which  will  be  randomly  selected  for  sampling.  The  aim  of 
the  basket  assignment  process  is  to  make  each  basket  resemble  the  population  in  any 
traits  of  interest.  In  general,  the  spread  and  proportion  of  proposal  values  will  be 
nearly  identical  to  the  population.  Sophisticated  software  can  quickly  balance  baskets 
for  work  center  assignments,  material  usage,  skill  requirements,  etc.  In  short,  whatever 
trait  is  deemed  pertinent  to  the  value  of  an  audit  will  be  balanced  by  the  Basket 
Method  where  possible.  Obviously,  in  a  population  of  proposals  where  one  is  ten  times 
larger  than  any  others  it  would  be  more  practical  to  remove  that  proposal  from  the 
population  before  assigning  the  baskets. 

1.  Basket  Assignment 

Assignment  of  a  population  to  baskets  follows  a  simple  set  of  rules: 

1.  All  the  proposals  in  the  population  are  arranged  in  descending  order  by  book 
value. 

2.  The  largest  unassigned  proposal  is  assigned  to  the  first  basket,  the  second 
largest  to  the  second  basket,  and  so  on  for  the  first  proposals. 

3.  The  baskets  are  sorted  in  ascending  order  by  total  assigned  proposal  book  value. 
(The  first  time  this  step  is  performed  it  will  simply  reverse  the  order  of  the 
baskets.) 

4.  The  assignment  and  sort  steps  are  repeated  until  all  the  proposals  are  assigned 
to  a  basket. 

The  effect  of  this  procedure  is  to  add  the  largest  of  the  next  n  unassigned 
proposals  to  the  basket  with  the  smallest  total  book  proposal  value.  The  end  result  is 
that  each  basket  will  have  nearly  identical  total  book  values  and  proposal  size  mixes. 
The  software  previously  noted  will  make  adjustments  to  correct  for  any  deviations  in 
totals  and  balance  the  mix  for  traits  other  than  cost  as  well. 
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2.  Evaluation  of  Baskets 

One  basket  is  chosen  randomly  and  all  its  proposals  are  audited.  The 
resulting  audit  value  of  the  basket  is  divided  by  the  proposal  value  to  produce  a 
correction  factor.  The  proposal  value  of  the  population  is  then  multiplied  by  the 
correction  factor  to  determine  the  population  audit  result.  This  will  be  the  estimated 
true  value  of  the  population. 
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IV.  DESCRIPTION  OF  SIMULATION 


A.  DERIVING  COMPARABLE  RESULTS 

Direct  comparison  of  the  Basket  Method  and  the  Dollar  Unit  Sampling  method 
will  not  be  meaningful  since  the  former  yields  a  best  estimate  of  the  error  rate  of  a 
population  and  the  latter  returns  an  upper  error  limit  for  a  stated  assurance  level. 
Since  it  is  the  consistency  of  the  drawn  sample  which  is  of  interest  in  this  investigation, 
the  sample  selection  process  of  both  the  Basket  Method  and  the  Dollar  Unit  Sampling 
Technique  will  be  used  to  draw  samples  from  the  same  populations.  The  samples  will 
then  be  evaluated  according  to  the  Basket  Method,  which  will  give  an  estimate  of  the 
true  value  of  the  population,  to  see  how  well  the  sample  produced  by  each  method 
reflects  the  value  of  the  population.  This  is  appropriate  since  the  negotiated  price  of  a 
contract  should  be  a  best  estimate,  not  an  upper  limit. 

The  rules  of  the  Basket  Method  will  create  the  identical  set  of  baskets  from  a 
given  population  every  time  they  are  applied.  Therefore,  the  baskets  of  the  Basket 
Method  can  easily  be  evaluated  by  complete  enumeration.  The  rules  of  the  Dollar 
Unit  Sampling  technique  also  provide  a  finite  number  of  samples,  but  that  number  is 
much  larger  than  the  number  of  different  baskets.  Therefore,  the  Dollar  Unit  Sampling 
technique  will  be  evaluated  using  a  Monte  Carlo  process  in  which  a  random  start 
number  is  generated  for  each  run  and  the  AS1  and  order  of  proposals  are  kept 
constant. 

To  evaluate  the  sample  drawn  with  the  Dollar  Unit  Sampling  rules  using  the 
Basket  Method  evaluation  technique  the  sample  is  treated  as  if  it  was  a  basket.  All  its 
resident  proposals  are  audited  and  their  true  value  is  divided  by  their  proposal  value. 
The  resulting  factor  is  multiplied  against  the  population  proposal  total  to  determine  the 
best  estimate  of  the  true  total  value  of  the  proposal  population. 

B.  PREPARATION  OF  THE  TEST  POPULATIONS 

The  populations  used  are  described  in  Table  1. 

The  original  populations  wrere  randomly  seeded  with  errors  at  a  5%  and  10% 
rate  of  occurance.  The  5%  population  was  then  skewed  to  form  two  additional  test 
populations.  One  had  its  errors  skewed  strongly  to  its  higher  valued  proposals,  the 
other  to  its  lower  valued  proposals.  The  total  dollar  amount  of  error  and  number  of 
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TABLE  1 

POPULATION  DESCRIPTION 


NAME 

A 

Population 

8,300 

Erroneous  Proposals 

5% 

£  S  Errors/^  S  Proposals 

9% 

Types  of  Errors 

+ 

Skew  (none,  high,  or  low) 

N 

NAME 

AA 

Population 

8,300 

Erroneous  Proposals 

5% 

£  $  Errors/^]  S  Proposals 

9% 

Types  of  Errors 

+  /- 

Skew  (none,  high,  or  low) 

N 

B 

C 

D 

E 

8,300 

8,300 

8,300 

8,300 

5% 

5% 

10% 

10% 

9% 

9% 

0.1% 

10% 

+ 

+ 

+ 

+ 

H 

L 

N 

N 

BB 

CC 

DD 

EE 

8,300 

8,300 

8,300 

8,300 

5% 

5% 

10% 

10% 

9% 

9% 

0.1% 

10% 

+  /- 

+  /- 

+  !- 

+  /- 

H 

L 

N 

N 

overstated  proposals  remained  constant  during  the  skewing  process.  All  populations 
were  created  in  both  a  "dishonest”  version  which  had  overstatements  only  and  are 
named  by  single  letters,  and  in  an  "honest"  version  with  both  overstatements  and 
understatements  named  by  double  letters.  Except  for  the  sign  on  each  error,  the  single 
letter  named  populations  are  identical  to  their  double  letter  named  counterparts.  The 
populations  with  10%  errors  differ  in  that  E  and  EE,  while  containing  the  same 
number  of  errors  in  the  same  distribution  and  sign  as  D  and  DD  respectively,  have 
errors  of  much  larger  magnitude,  so  that  the  sum  of  the  dollar  value  of  the  errors  make 
up  10%  of  the  population  in  E  and  EE  but  only  0.1%  of  the  population  in  D  and  DD. 

C.  SIMULATION  EXECUTION 

Simulations  were  run  on  all  populations  using  both  a  Basket  Method  evaluating 
program  and  a  Dollar  Unit  Sampling  monte  carlo  simulation  program.  First,  ten 
baskets  were  chosen  for  the  Basket  Method  to  remain  consistent  with  the  earlier 
arbitrary  decision  to  audit  one  tenth  of  the  proposals.  This  resulted  in  830  proposals 
per  basket.  Next,  an  ASI  was  then  selected  to  produce  an  average  of  830  proposals  in 
the  Dollar  Unit  Sampling  simulation,  and  ten  trials  were  run  using  the  Dollar  Unit 
Sampling  selection  method.  The  programs,  written  for  this  study  in  Waterloo  Basic, 
are  listed  in  Appendix  A. 
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The  ten  audit  results  for  each  method  were  then  divided  by  their  respective 
proposal  sums  to  derive  the  correction  factors.  These  correction  factors  were 
multiplied  by  the  sum  of  all  proposals  to  give  the  predicted  true  total  for  the 
population.  The  difference  between  the  predicted  and  the  actual  total  was  then  divided 
by  the  sum  of  all  proposals  to  give  a  percent  error  for  each  basket  and  trial.  The 
means  of  the  percent  errors  for  each  method  and  each  population  are  listed  in  Table  2. 


TABLE  2 

SIMULATION  RESULTS  (MEAN  %  ERROR) 


NAME 

A 

B 

C 

D 

E 

Basket  Method 

0.77 

1.03 

0.66 

0.14 

0.35 

DUS 

2.40 

-2.06 

2.75 

0.25 

4.75 

NAME 

AA 

BB 

CC 

DD 

EE 

Basket  Method 

0.99 

0.99 

1.16 

0.15 

1.21 

DUS 

0.13 

0.14 

-0.01 

-0.08 

-0.28 

Detailed  results  are  shown  in  Appendix  B.  A  positive  percent  error  represents  an 
overestimate  and  a  negative  number  represents  an  underestimate  error.  Using  the 
Basket  Method  to  draw  a  sample  was  always  more  accurate  against  the  overstatements 
only  padding  technique.  Using  the  Dollar  Unit  Sampling  technique  to  draw  the  sample 
was  more  accurate  when  the  proposals  contained  errors  in  both  directions. 

D.  RESISTANCE  TO  PROPOSAL  RIGGING 

In  order  to  benefit  from  the  potential  time  and  labor  savings  a  sampling  system 
offers,  the  sampling  technique  must  be  resistant  to  padding  schemes.  If  not,  a 
contractor  has  much  to  gain  by  manipulating  the  errors  in  his  proposals.  Therefore,  as 
mentioned  previously,  the  central  issue  is  not  necessarily  to  determine  which  of  these 
two  methods  is  the  most  accurate,  but  to  discover  which  one  benefits  padding  schemes 
least.  When  comparing  a  method's  performance  between  the  single  and  double  letter 
versions  of  a  population  it  can  be  seen  that  the  Basket  Method  is  essentially  as  strict  or 
stricter,  from  the  governments  view,  when  estimating  the  value  of  the  overstatement 
only  population  as  it  is  when  estimating  the  value  of  the  honest  population.  For 
example,  the  A  population,  which  was  rigged  with  overstatements  only,  was  evaluated 
by  the  Basket  Method  at  a  value  exceeding  its  actual  value  by  0.77%.  The  AA 
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population,  with  randomly  decided  over  and  understatements,  was  evaluated  at  a  value 
exceeding  its  true  value  by  0.99%.  If  these  two  populations  had  been  submitted,  the 
contractor  who  made  overstatements  only  would  have  received  0.22%  less  in  payment 
from  the  government.  In  other  words,  padding  one's  proposed  contract  changes  with 
overstatements  in  random,  low,  or  high  skewed  distributions  prior  to  submitting  them 
to  the  Basket  Method  for  evaluation  is  not  likely  to  raise  the  resulting  estimate  for  the 
population  but  is  instead  likely  to  lower  the  estimated  value.  On  the  other  hand,  the 
sample  drawn  with  Dollar  Unit  Sampling  allowed  the  overstatement  padded 
populations  a  significantly  larger  estimate  than  it  did  the  honest  populations. 

E.  EVALUATION 

Assuming  honest  contractors  are  as  likely  to  understate  as  overstate  their  costs 
and  dishonest  contractors  are  not,  honest  contractors  will  be  more  successful  than 
dishonest  contractors  against  the  Basket  Method,  and  dishonest  contractors  will  be 
more  successful  than  honest  contractors  against  the  Dollar  Unit  Sampling  sample 
method.  To  choose  the  Dollar  Unit  Sampling  method  to  determine  a  contract  value 
would,  in  effect,  encourage  overbidding,  since  it  would  be  more  profitable  than 
remaining  honest.  Under  the  Basket  Method  honesty  is  as  profitable  as  dishonesty. 

F.  AREAS  FOR  FURTHER  RESEARCH 

This  study  may  be  expanded  in  several  ways.  It  may  be  repeated  with  fewer  or 
more  baskets.  A  data  set  with  a  much  smaller  variance  in  proposal  size  may  be  used. 
The  Basket  Method  can  be  compared  to  additional  sampling  methods.  Additional 
strategies  for  error  arrangements  which  will  reduce  the  accuracy  of  the  Basket  Method 
can  be  developed  and  tested.  Particularly,  will  an  understatement  only  strategy  with 
the  same  magnitude  and  distribution  of  proposal  errors  result  in  a  total  population 
error  in  equal  magnitude  and  opposite  sign  as  an  overstatement  strategy. 


18 


APPENDIX  A 

SIMULATION  PROGRAM  LISTINGS 


00100 

00120 

00140 

00160 

00180 

00200 

00220 

00240 

00260 

00280 

00300 

C0320 

00340 

00360 

00380 

00400 

00420 

00440 

00460 

00480 

00500 

00520 

00540 

00560 

00580 

00600 

01000 

01020 

01040 

01060 

01080 

01100 

01120 

01140 

01160 

01180 

02000 

02020 

02040 

02060 

02080 

02100 

02120 

02140 

02160 

02180 

02200 

02220 

02240 

02260 

02280 

02300 

02320 

02340 

02360 

02380 

03000 

03020 

03040 


Basket  Method  Enumeration  Program  Listing 

REM  THIS  IS  A  PROGRAM  TO  PROCESS  DATA  USING  THE  BASKET  METHOD 
REM  DATA  IS  INPUT  FROM  A  FILE.  SEPARATED  BY  COMMAS,  AND  LISTED 
REM  AS  PAIRS  OF  VALUES  FOR  A  SlD.  THE  BOOK  FIRST  AtiD  THE 
REM  AUDITED  VALUE  SECOND.  THE  PROGRAM  EXPECTS  1 DATPOP '  PAIRS 
REM  OF  VALUES.  DATA  MUST  BE  IN  DESCENDING  ORDER  BY  BOOK  VALUE. 
REM 

REM  **  DIMENSION  VARIABLES  ** 

DIM  ASUM( 50) ,  BSUM( 50) ,  ANEXT(50).  BNEXT(50) 

DIM  ERRORP( b6) ,  FACTOR(50),  ERRORA(50) 

REM 

REM  **  SET  CONSTANTS  ** 

REM 


B  =  10 
DATPOP  = 
BPOP= 


8300 

POP=INT(DATPOP/B)  !  I 
OPEN  #3,  'TEST  (RECFM 
ATOT  =  0 


NUMBER  OF  BASKETS 
NUMBER  OF  DATA  PAIRS 
INITIATE  RUNNING  TALLY  OF 
F  LRECL  80)' ,  INPUT 


DATA  PAIRS  READ 


SUM  OF 
SUM  OF 


BTOT  =  0 
BPOP1  =  1 
FOR  J  = 

ASUMtJ 
BSUM(J 
NEXT  J 
EES  =  0 
EED  =  0 
REM 

REM  **  ROUTINE  TO  READ  IN  DATA  ** 
REM 

IF  BPOP1  >  BPOP 
GOTO  4000 
ENDIF 
FOR  I  =  1  TO  B 

INPUT  #3,  ANEXT(I) ,  BNEXT(I) 

NEXT  I 

BP0P1  =  BPOP1  +  1 
REM 

REM  **  ROUTINE  TO  SORT  PARTIAL  SUMS 
REM  **  BASKETS  IN  ASCENDING  ORDER 
REM 


ERROR  SQUARES 
BASKET  DOLLAR  SQUARES 


IF  NO  MORE  DATA,  THEN  PROCESS 


IN 


** 


I  =  1 
WHILE 
IF 


I 


I  < 

8SUM 
Cl  = 

C2  = 

BSUM 
ASUM 
BSUM 
ASUM 
IF  I 
I  _ 
ENDIF 
GOTO  2120 
ENDIF 
=  1+1 


BSUM( 1+1) 


UM 


I)  =  B 
I)  =  ASUM 
1+1)  =  Cl 
1+1)  =  C2 
>  1 
=  1-1 


m) 


REM 

REM 

REM 


ENDLOOP 

**  ADD  NEXT  ROUND  TO  BASKETS 
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03060 

03080 

03100 

03120 

03140 

04000 

04020 

04040 

04060 

04080 

04100 

04120 

04140 

04160 

04180 

04185 

04190 

04200 

04220 

04240 

04260 

04280 

04300 

05000 

05020 

05040 

05060 

05080 

05100 

05120 

05140 

05160 

05180 

05200 

05220 

05240 

05260 

05280 

05300 

06000 

06020 

06040 

06060 

C6080 

06100 

06120 

07000 


FOR  I  =  1 
BSUM  " 
ASUM 
NEXT  _ 
GOTO  1060 
REM 
REM 
REM 

FOR  I  = 


(I)  =  BSUMfl 

)  +  BNEXT(I) 

(I)  =  ASUM( I 

)  +  ANEXT(I) 

ADDING  ROUTINE 


_  1  TO  B 

BTOT  =  BTOT 
ATOT  =  ATOT 
I 

1  TO 


BSUM(I) 

ASUM(I) 


B 


NEXT 

FOR  I  =  . 

FACTOR! I 
ERRORACI 
EES  =  EE 
EED  =  EEV 
ERRORPm  : 

MAE  =  MAE 
MPE  =  MPE 
NEXT  I 
MAE  =  MAE  /  B 
MPE  =  MPE  /  B 
REM 

REM  **  PRINT  RESULTS  * 
REM 

PRINT  'BASKET  BOOK  VALUE 

formo$=ytotal  #######.  ## 

FORMi:;=  ##  #######.  ## 

F0RM2‘»='  MEAN 
PRINT  USING  FORMOS,  BTOT, 
&  BTOT  -  ATOT 

FOR  I  =  1  TO  B 

PRINT  USING  F0RM1$ ,  I, 
&  ERRORACI ) 

NEXT  I 

PRINT  USING  F0RM2$ , 
F0RM3$-  DOLLARS 
F0RM4$=' CONTRACTS 
PRINT  ' 

PRINT  USING  F0RM3 
PRINT  USING  FORMA 
REM 

REM  **  CLEANUP 
REM 

CLOSE  #3 
END 


=  ASUM( I)/BSUM(I) 

=  BTOT  *  FACTOR? I)  -  ATOT 


ERRORA(I)  *  ERRORA(I) 
BSUM(I)  *  BSUM(I) 

100  *  ERRORAf I;/BTOT 
ABS(ERRORAfl) 1 
ABS(ERRORP(I)) 


AUDIT  VALUE 

mum.  u 
mm#.  ## 


FACTOR  %ERROR  ERR  OF  PR' 
#.####  ##.####  #####.##' 
#.####  #.####  #####.  ## 
k  uuuu  uauuti  tttt ' 
ATOT,  ATOT/BTOT,  100*(BTOT-ATOT)/BTOt!& 


BSUM(I),  ASUM(I) ,  FACTOR( I ) ,  ERRORP(I),& 


MPE,  MAE 

#U####.  ## 
#######.  ## 
MEAN  AUDITED 


#######.  ##\ 

#######.##' 

S  D 

, BTOT/B , SQR( ((B*EED)-(BTOT*BTOT))/(B*(B-l) ) ) 

;datpop;b,o 
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00010 

00100 

00120 

00140 

00160 

00180 

00200 

00220 

00240 

00260 

01000 

01020 

01040 

01060 

01080 

01100 

01120 

01140 

01160 

01180 

01200 

01220 

01240 

01260 

01280 

01300 

01310 

01320 

01340 

01360 

01380 

01400 

01420 

01440 

01460 

01480 

02000 

02020 

02040 

02060 

02080 

02100 

02120 

02140 

02160 

02180 

02200 

03000 

03020 

03040 

03060 

03080 

03100 

03120 

03140 

03160 

03280 

03300 

03320 

03340 

04000 

04020 

04040 

04060 

04080 

04100 

04120 


Dollar  Unit  Sampling  Monte  Carlo  Program  Listing 

EM  ******  DUS  USING  ASI  ON  TESTSET  ****** 

EM  THIS  IS  A  PROGRAM  TO  PROCESS  DATA  USING  DOLLAR  UNIT  SAMPLING. 

EM  DATA  IS  INPUT  FROM  A  FILE.  SEPARATED  BY  COMMAS,  AND  LISTED 
EM  AS  PAIRS  OF  VALUES  FOR  A  6lDA  THE  BOOK  FIRST  AND  THE 
EM  AUDITED  VALUE  SECOND.  THE  PROGRAM  EXPECTS  ' DATPOP '  PAIRS 
EM  OF  VALUES.  DATA  MAY  BE  IN  ANY  ORDER 
EM 

EM  **  SET  CONSTANTS  ** 

EM 

!  RANDOMIZE  THE  START  OF  THE  RANDOM  NUMBER  GEN 


RANDOMIZE 

REM 

REM  **" 
REM 
=  0 
1  =  1 
EA 


SET  THESE  VALUES  FOR  EACH  NEW  FILE 


*  rtrt* 


EP 
EES 
EEL 
EEV 
EC  =  0 
EC2  =  0 
ED  =  0 
ED2  =  0 


B  =  21 

DATPOP  =  8300 
ASI  =  DATPOP  /  B 
OPEN  #3,  'TEST  (RECFM  F  LRECL  80)',  INPUT 
POP  =  0 
AUDITSUM  =  0 
RESULTS  =  0 
BIDTOTAL  =  0 
TRUETOTAL=  0 
ECC  =  0 
K  =  RND  *  ASI 
RTALLY  =  -K 
REM 

REM  **  ROUTINE  TO  READ  IN  DATA  ** 

REM 


LAP  COUNTER 

TOTAL  LAPS  TO  BE  RUN 

SUM  OF  ACTUAL  ERROR 

SUM  OF  PERCENTAGE  ERROR 

SUM  OF  ERROR  SQUARES 

SUM  OF  PREDICTIONS 

SUM  OF  AUDIT  SIZE  IN  DOLLARS 

NUMBER  OF  CONTRACTS  AUDITED 

EC-SQUARED 

TOTAL  BID  DOLLARS  IN  CONTRACTS  AUDITED 
ED-SQUARED 

NUMBER  OF  BASKETS  USED  IN  ALT  METHOD 
NUMBER  OF  DATA  PAIRS 


POP  =  POP  +  1 
IF  POP  >  DATPOP 
GOTO  4000 
ENDIF 
INPUT  #3,  NEXTAUD. 


IF  NO  MORE  DATA,  THEN  PROCESS 


NEXTBID 


BIDTOTAL  =  BIDTOTAL  +  NEXTBID 
TRUETOTAL  =  TRUETOTAL  +  NEXTAUD 
RTALLY  =  RTALLY  +  NEXTBID 
REM 

REM  **  ROUTINE  TO  DETERMINE  IF 
REM  **  THIS  BID  WILL  BE  AUDITED 
REM 

IF  RTALLY  <  0 
GOTO  2060 
_ENDIF 

RTALLY  =  RTALLY  -  (R  *  ASI) 
AUDITSUM  =  AUDITSUM  +  NEXTBID 
RESULTS  =  RESULTS  +  NEXTAUD 
ECC  =  ECC  +  1 
GOTO  2060 

REM  **  FINAL  CALCULATIONS  ** 

REM 

F  =  TRUETOTAL/BIDTOTAL 

AE  =  BIDTOTAL  -  TRUETOTAL 

FF  =  RESULTS  /  AUDITSUM 

EA  =  (BIDTOTAL  *  FF)  -  TRUETOTAL 

EP  =  EA  *  100  /  BIDTOTAL 
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04140 

04160 

04180 

04200 

04220 

04240 

04260 

04280 

04300 

05000 

05020 

05040 

05060 

05080 

05100 

05101 

05102 

05120 

05140 

05160 

05180 

05200 

05220 

05240 

05260 

05280 

06000 

06020 

06040 

06060 

06080 

06100 

06120 

06140 

06160 

06180 

06190 

06200 

06220 

06240 

06260 

06280 

06300 

06320 

06340 

06360 

06380 

07000 


EEA  =  EEA 
EEP  =  EEP 
EES  =  EES 
EEL  =  EEL 
EEV  =  EEV 
EC  =  EC  + 

EC2  =  EC2 
ED  =  ED  + 

ED2  =  ED2 
REM 
REM 
REM 
FORMO 
F0RM1 
F0RM2 
F0RM3 
FORM  4 
IF  L  =  . 

PRINT  'BASKET 


EA 

EP 

EA  *  EA 

BIDTOTAL 

AUDITSUM 


FF 


ECC) 


ECC  * 
ITSUM 
AUDITSUM 


**  PRINT  RESULTS 


AUDITSUM) 


TOTAL  #######.  ##  #######.  ##  #.  ####  M #.  ###  MMM.  M 

AUDIT  MMM#.  M  #######.##  #.####  ###.###  ######.## 

#######.##  MMM#.  M  ######.###  #######.  ###{ 
DOLLARS  #########.##  #########.  ##\ 

CONTRACTS  #########.##  #########.##' 


_  .  .  BOOK  VALUE  AUDIT  VALUE  FACTOR  %ERR0R  ABS  ERROR 
PRINT  USING  FORMOS,  BIDTOTAL,  TRUETOTAL,  F,& 

&  100*(AE)/BIDTOTA£.,& 


&  AE 
ENDIF 

PRINT  USING  F0RM1$ 

-  - HTAI  * 


((BIDTOTAL 
FF) 


&  100 

&  (BIDTOTAL  * 

REM 

REM  **  CLEANUP  * 
REM 

CLOSE  #3 
L  =  L  +  1 
IF  L  <  LI 
GOTO  1320 
ENDIF 

PRINT  '  MEAN  PRED 
USING  FORM2$ , 


AUDITSUM.  RESULTS,  FF.& 

FF)  -  TRU£TOTAL)/BiDTOtAL,& 
TRUETOTAL 


PRINT 
PRINT 
PRINT  ' 

PRINT  USING  FORM3S 
PRINT  USING  FORM43 
PRINT 

PRINT  'HOW  MANY  MORE' 
INPUT  L2 
LI  =  LI  +  L2 
IF  L2  >  0 
GOTO  1320 
ENDIF 
END 


S.  D. 


MEAN  %  ERR  MEAN  ERROR' 


EEL/LI ,  SQR( EES/( Ll-1) ) , EEP/L1 ,  EEA/L1 


MEAN  AUD 
, ED/ LI  "■ 
EC/Ll 


Ll*ED2)-( 
L1*EC2 
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APPENDIX  B 
DETAILED  RESULTS 


POPULATION  A 


BASKET 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605. 73 

372255. 73 

0.  9088 

37350.  00 

1 

40960. 47 

36910. 47 

0. 9011 

-00. 77 

-3150. 10 

2 

40960. 51 

38170.  51 

0. 9319 

002. 31 

9449. 96 

3 

40960. 57 

37450. 57 

0. 9143 

000. 55 

2250. 00 

4 

40960. 57 

37540. 57 

0.9165 

000. 77 

3150.00 

5 

40960. 55 

37270. 55 

0. 9099 

000. 11 

449.  98 

6 

40960. 56 

36370. 56 

0.8879 

-02.  09 

-8550. 01 

7 

40960. 57 

37090. 57 

0. 9055 

-00. 33 

-1350. 00 

8 

40960. 66 

37270. 66 

0. 9099 

000. 11 

450.08 

9 

40960. 64 

37090.  64 

0. 9055 

-00. 33 

-1349. 94 

10 

40960. 63 

37090.63 

0.9055 

-00. 33 

-1349. 95 

MEAN 

000. 77 

3150. 00 

MEAN 

AUDITED 

S.  D. 

DOLLARS  40960. 57 

0.06 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERR0R 

Numeric  Error 

TOTAL 

409605. 73 

372255.  73 

0.9088 

37350. 00 

AUDIT 

158976. 31 

148626.  31 

0. 9349 

2.  608 

10683.01 

AUDIT 

158949. 19 

148599.  19 

0.9349 

2.  607 

10678. 46 

AUDIT 

158824. 04 

147574.  04 

0. 9292 

2.  035 

8336. 35 

AUDIT 

158604. 10 

148614.  10 

0. 9370 

2.820 

11550. 15 

AUDIT 

158856. 17 

147966.  17 

0. 9314 

2.  263 

9270. 47 

AUDIT 

158996. 13 

149096. 13 

0.  9377 

2.892 

11845. 63 

AUDIT 

158693. 67 

147443.  67 

0.  9291 

2.  029 

8312. 52 

AUDIT 

158827.85 

147757.85 

0. 9303 

2.  149 

8801.  26 

AUDIT 

158825. 64 

147755. 64 

0.  9303 

2.  149 

8800.86 

AUDIT 

159021. 75 

148401.  75 

0. 9332 

2.440 

9995. 17 

MEAN 

PRED 

S.  D. 

MEAN  %  ERR 

MEAN  ERROR 

382083. 12 

10441. 56 

2.399 

9827. 389 

DOLLARS 

CONTRACTS 


MEAN  AUDITED 
158857.48 
829.80 


S.  D. 
134.  49 
0.42 
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POPULATION  B 


BASKET  BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL  409605. 73 

372255. 73 

0. 9088 

37350. 00 

1 

40960. 52 

37270. 52 

0. 9099 

000. 11 

449. 95 

2 

40960. 53 

36820. 53 

0.8989 

-00. 99 

-4050. 04 

3 

40960. 56 

37630. 56 

0. 9187 

000.99 

4049.99 

4 

40960. 55 

37000. 55 

0. 9033 

-00.  55 

-2250. 02 

5 

40960. 57 

36550. 57 

0.8923 

-01.65 

-6750.00 

6 

40960. 61 

36910. 61 

0.  9011 

-00.77 

-3149. 96 

7 

40960. 62 

37450.  62 

0.  9143 

000. 55 

2250.04 

8 

40960. 60 

38080. 60 

0. 9297 

002.09 

8550.  02 

9 

40960. 61 

36730. 61 

0.8967 

-01.21 

-4949.96 

10 

40960. 56 

37810. 56 

0.9231 

001.  43 

5849.  99 

MEAN 

001.  03 

4230.  00 

MEAN 

AUDITED 

S.  D. 

DOLLARS  40960. 57 

0.  04 

CONTRACTS 

830. 00 

0.  00 

DUS 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTA 

409605.  73 

372255. 73 

0.9088 

37350. 00 

AUDI' 

171892.21 

152812. 21 

0. 8890 

-1.  981 

-8116. 15 

AUDI' 

171815.30 

152735. 30 

0.8890 

-1.  986 

-8136.  50 

AUDI' 

172017.42 

151497. 42 

0.8807 

-2.810 

-11511.  97 

AUDI' 

171892.21 

152812. 21 

0.8890 

-1.  981 

-8116. 15 

AUDI' 

172155.89 

153435.89 

0.8913 

-1.  755 

-7189. 98 

AUDI' 

172040.  59 

152150. 59 

0. 8844 

-2.443 

-10005. 44 

AUDI' 

171784.85 

153334.85 

0.8926 

-1.622 

-6642. 39 

AUDI' 

172091.02 

152651. 02 

0.8870 

-2. 178 

-8920. 49 

AUDI' 

172171.50 

153361. 50 

0.  8907 

-1.  807 

-7400. 05 

AUDI' 

171894.96 

152814.96 

0.8890 

-1.  981 

-8115.  42 

MEAN  PRED 
363840. 28 


DOLLARS 

CONTRACTS 


S.  D. 
8985.  13 

MEAN  AUDITED 
171975. 59 
830.00 


MEAN  %  ERR 
-2.055 


MEAN  ERROR 
-8415.  455 


S.  D. 
138. 51 
0.  00 
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POPULATION  C 


BASKET 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605.73 

372255. 73 

0. 9088 

37350. 00 

1 

40960.48 

37090.  48 

0.9055 

-00.  33 

-1350.09 

2 

40960. 53 

37540. 53 

0.9165 

000.77 

3149. 96 

3 

40960. 57 

37000. 57 

0.9033 

-00.55 

-2250.00 

4 

40960.54 

36910. 54 

0. 9011 

-00. 77 

-3150. 03 

5 

40960. 57 

36910. 57 

0.9011 

-00. 77 

-3150.00 

6 

40960. 58 

36910. 58 

0.9011 

-00.77 

-3149. 99 

7 

40960. 59 

37720. 59 

0.  9209 

001.  21 

4950.01 

8 

40960. 64 

37180. 64 

0.9077 

-00. 11 

-449. 94 

9 

40960.64 

37630. 64 

0. 9187 

000. 99 

4050.05 

10 

40960. 59 

37360. 59 

0. 9121 

000.  33 

1350. 01 

MEAN 

000. 66 

2700.  01 

DOLLARS 

CONTRACTS 


MEAN  AUDITED 
40960. 57 
830.  00 


DUS 

BOOK  VALUE 
TOTAL  409605. 73 
AUDIT  157773. 16 
AUDIT  157713.97 
AUDIT  157738.  10 

AUDIT  158027.  14 

AUDIT  157816.30 
AUDIT  157838.  59 
AUDIT  158054.38 
AUDIT  157816.61 
AUDIT  157782.  93 
AUDIT  157840.97 
MEAN  PRED 
383541. 10 


AUDIT  VALUE 
‘72255.  73 
47783. 16 

48353.97 
48198. 10 
48217. 14 
47646. 30 
47398. 59 
47794.38 
47556. 61 
47702. 93 

47310. 97 
S.  D. 

11935. 61 


DOLLARS 

CONTRACTS 


MEAN  AUDITED 
157840.21 
831.00 


S.  D. 
0.  05 
0.  00 


FACTOR  %ERROR 
0. 9088 

0.9367  2.787 

0.9407  3.184 

0.9395  3.071 

0.9379  2.911 

0.9356  2.674 

0.  9339  2.  504 

0.9351  2.627 

0.9350  2.617 

0.9361  2.730 

0.9333  2.447 

MEAN  %  ERR 
2.755 

S.  D. 

113. 57 
0.00 


Numeric  Error 
37350.00 
11414. 28 
13040. 74 
12577. 05 
11922. 52 
10954. 18 

10257. 24 
10760. 70 
10720. 64 

11182. 24 
10024. 09 

MEAN  ERROR 
11285. 367 
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POPULATION  D 


BASKET 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric 

TOTAL 

375545. 95 

372255. 73 

0.9912 

3290.  22 

1 

37503.29 

37274. 23 

0.9939 

0. 2653 

996.49 

2 

37507.89 

37282. 36 

0.9940 

0. 2748 

1032.  11 

3 

37520. 51 

37172. 52 

0.9907 

-.0513 

-192.84 

4 

37530.36 

37232. 51 

0. 9921 

0.0825 

309.80 

5 

37540.  11 

37167.81 

0.9901 

-. 1156 

-434. 22 

6 

37571. 67 

37233. 59 

0.9910 

-. 0237 

-89.  04 

7 

37580. 30 

37271. 16 

0.9918 

0. 0535 

200.93 

8 

37589.90 

37164. 13 

0.9887 

-. 2566 

-963.48 

9 

37591. 16 

37249. 01 

0.9909 

-.0341 

-127.95 

10 

37610. 76 

37208. 41 

0. 9893 

-. 1937 

-727. 27 

MEAN 

0. 1351 

507. 41 

MEAN 

AUDITED 

S.  D. 

DOLLARS  37554. 59 

38.  68 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric 

TOTAL 

375545.95 

372255. 73 

0.  9912 

0.876 

3290.22 

aud: 

it 

154991.06 

154005.  19 

0.9936 

0.240 

901.44 

aud: 

IT 

155019.49 

154034. 41 

0.  9936 

0.  241 

903.  79 

aud: 

IT 

154681.74 

153707.70 

0.  9937 

0.246 

925. 39 

aud: 

IT 

154986. 95 

154047. 14 

0.  9939 

0.270 

1012. 98 

aud: 

IT 

155019.33 

154032. 08 

0. 9936 

0.  239 

898. 53 

aud: 

IT 

154677.71 

153762. 34 

0.9941 

0.284 

1067. 77 

aud: 

IT 

154698.02 

153716. 51 

0.9937 

0.242 

907. 50 

aud: 

IT 

155015.48 

154063. 07 

0.9939 

0.262 

982.88 

aud: 

IT 

154917. 86 

153974. 62 

0.9939 

0.267 

1003.  65 

aud: 

IT 

154721.97 

153729. 01 

0.  9936 

0.  234 

880.08 

MEAN  PRED 
373204.  13 


DOLLARS 

CONTRACTS 


S.  D. 
1001.71 

MEAN  AUDITED 
154872.  96 
838. 10 


MEAN  %  ERR 
0.253 


MEAN  ERROR 
948. 401 


S.  D. 
156. 44 
0.  32 
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POPULATION  E 


BASKET 

BOOK  VALUE  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

413614.33  372255.73 

0. 9000 

41358. 60 

1 

4] 

1361.  43 

37265. 53 

0. 9010 

000. 10 

399. 60 

2 

4] 

L361.  45 

36965.85 

0.8937 

-00. 63 

-2597. 38 

3 

4] 

L361.  39 

37215. 54 

0.8998 

-00. 02 

-99.  94 

4 

4] 

1361.  39 

36765. 99 

0. 8889 

-01.  11 

-4595.  45 

5 

4] 

L361. 39 

37265. 49 

0.9010 

000. 10 

399. 56 

6 

41 

L361.  37 

37465.27 

0.9058 

000. 58 

2397. 54 

7 

41 

1361. 40 

37365. 40 

0. 9034 

000.  34 

1398. 57 

8 

41 

L  36 1 . 50 

37365. 50 

0. 9034 

000.  34 

1398. 66 

9 

41 

L361.  50 

37315. 55 

0. 9022 

000.22 

899. 17 

10 

41 

L361.  51 

37265.  61 

0. 9010 

000. 10 

399. 68 

MEAN 

000. 35 

1458. 55 

MEAN  AUDITED 

S.  D. 

DOLLARS 

41361. 

43 

0.05 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTA 

413614.33  372255.73 

0.9000 

41358. 60 

AUDI' 

155856.65  1 

L47964. 55 

0. 9494 

4.  936 

20414. 45 

AUDI' 

155788.89  1 

147746.  94 

0. 9484 

4.837 

20007. 49 

AUDI' 

155686.91  1 

146995. 61 

0. 9442 

4.  417 

18268.  37 

AUDI' 

156020.28  1 

[47578. 73 

0. 9459 

4.  589 

18979.80 

AUDI' 

155820.19  1 

[47678. 34 

0. 9477 

4.  774 

19746. 60 

AUDI' 

155941.85  1 

L 48299. 50 

0. 9510 

5.  099 

21088. 32 

AUDI' 

155747.96  1 

[47306. 41 

0. 9458 

4.  579 

18940. 68 

AUDI' 

155728.97  1 

[46937. 77 

0.9435 

4.  354 

18009. 28 

AUDI' 

155957.70  1 

[48265. 40 

0.9507 

5.067 

20957. 90 

AUDI' 

155987.63  1 

[47995. 63 

0. 9488 

4.876 

20167. 14 

MEAN 

PRED 

S.  D. 

MEAN  %  ERR 

MEAN  ERROR 

391913. 73 


DOLLARS 

CONTRACTS 


20749.01 

MEAN  AUDITED 
155853.70 
830. 50 


4.753  19658.002 


S.  D. 
117.46 
0.  53 
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POPULATION  AA 


BASKET 

TOTAL 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

MEAN 


BOOK  VALUE 
409605. 73 
40960. 47 
40960. 51 
40960. 57 
40960. 57 

40960. 55 

40960. 56 

40960. 57 
40960. 66 
40960. 64 
40960.  63 


AUDIT  VALUE 
408975.73 
41050. 47 
40330. 51 

40690. 57 

39880. 57 

41410. 55 

41410. 56 

40870. 57 
41590. 66 
41050.64 
40690.  63 


DOLLARS 

CONTRACTS 


MEAN  AUDITED 
40960. 57 
830. 00 


DUS 

TOTAL 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 


MEAN  PRED 
409488.  74 


BOOK  VALUE 
409605.73 
158582.04 
158560.98 
158265.77 
158495.  01 
158599.  40 
158595.  41 
158446.91 
158327. 61 
158426.34 
158583.  44 


DOLLARS 

CONTRACTS 


MEAN  AUDITED 
158488.29 
825.90 


FACTOR 
0. 9985 
1.0022 
0. 9846 
0. 9934 
0.9736 
1.0110 
1.0110 
0. 9978 
1.  0154 
1.  0022 
0. 9934 

S.  D. 

0.06 

0.  00 


%ERR0R 

0. 3735 
-1.  3843 
-. 5054 
-2. 4829 
1.2524 
1.2524 
-.  0659 
1.  6919 
0.3735 
-. 5054 
0.9888 


AUDIT  VALUE 
408975. 73 
158582.04 
159100. 98 
157905. 77 
158855.01 

159049.40 

159315. 41 
158176.91 
157697. 61 
157256. 34 
158493.  44 

S.  D. 
1610.08 


Numeric  Error 
630.00 
1530.00 
-5670.01 
-2070.00 
-10170.00 
5130.00 
5130.00 
-270.00 
6929.99 
1530.00 
-2070.00 
4050.00 


FACTOR  %ERR0R 
0.9985 

1.0000  0.154 

1.0034  0.494 

0.9977  -.074 

1.0023  0.381 

1.0028  0.438 

1.0045  0.608 

0.9983  -.017 

0.9960  -.244 

0. 9926  -. 585 

0.9994  0.097 

MEAN  %  ERR 
0. 125 

S.  D. 

119.20 

0.32 


Numeric  Error 
630.00 
630.00 
2024.  97 
-301.  71 
1560.  36 
1792.  19 
2489.  55 
-67.98 
-999.86 
-2394. 99 
397. 54 
MEAN  ERROR 
513.006 


28 


POPULATION  BB 


BASKET 

TOTAL 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
MEAN 


BOOK  VALUE 
409605. 73 

40960. 52 

40960. 53 

40960. 56 

40960. 55 

40960. 57 

40960. 61 

40960. 62 

40960. 60 

40960. 61 

40960. 56 


AUDIT  VALUE 
410235. 73 

41230. 52 

40960. 53 

41590. 56 

40600. 55 

40690. 57 

40510.61 

40330. 62 

41860.60 

41410. 61 

41050.56 


DOLLARS 

CONTRACTS 


MEAN  AUDITED 
40960. 57 
830. 00 


DUS 

TOTAL 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 


MEAN  PRED 
410826.80 


DOLLARS 

CONTRACTS 


BOOK  VALUE 

409605. 73 
172018. 41 

171894. 73 
171856. 38 
172016. 66 
172023. 91 
172135.86 
171870. 32 
171851. 51 
172035. 15 
172155. 25 


FACTOR 
1.  0015 
1.  0066 
1. 0000 
1. 0154 
0. 9912 
0. 9934 
0. 9890 
0. 9846 
1.  0220 
1.0110 
1. 0022 

S.  D. 

0.04 

0.  00 


%ERROR 

0. 5054 
-. 1538 

1.  3843 
-1.  0327 

-.8130 
-1.2524 
-1.  6919 

2.  0434 
0.  9448 
0. 0659 
0.  9888 


AUDIT  VALUE 

410235. 73 
171208.41 

172524. 73 
172126. 38 
171386.66 
171753.91 
174835.86 
172410.32 
172391. 51 
171855. 15 
174495.  25 

S.  D. 

2855.  92 


Numeric  Error 
-630. 00 
2070. 00 
-630. 00 
5670. 00 
-4230. 00 
-3330. 00 
-5130. 00 
-6929. 99 
8369. 99 
3870. 00 
270. 00 
4050. 00 


MEAN  AUDITED 
171985.82 
830. 00 


FACTOR  %ERROR 
1.  0015 

0. 9953  -. 625 

1.0037  0.213 

1.0016  0.003 

0. 9963  -. 520 

0.9984  -.311 

1.0157  1.415 

1.0031  0.160 

1.0031  0.160 

0.9990  -.258 

1.0136  1.205 

MEAN  %  ERR 
0.  144 

S.  D. 

112. 32 
0.  00 


Numeric  Error 
-630. 00 
-2558. 75 
871. 22 
13.  52 
-2130. 15 
-1272. 90 
5794.  78 
656. 94 
657. 08 
-1058. 57 
4937. 52 
MEAN  ERROR 
591.  070 
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POPULATION  CC 


BASKET  BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Erro 

TOTAL  409605. 73 

410595. 73 

1.  0024 

-990. 00 

1 

40960.48 

40510.48 

0. 9890 

-1.3403 

-5490.01 

2 

40960. 53 

41140. 53 

1.0044 

0. 1978 

810.00 

3 

40960. 57 

40600. 57 

0. 9912 

-1. 1206 

-4590.00 

4 

40960. 54 

41770. 54 

1.0198 

1. 7358 

7110.01 

5 

40960. 57 

41590. 57 

1.  0154 

1. 2964 

5310.00 

6 

40960. 58 

41590. 58 

1.  0154 

1.  2964 

5310.00 

7 

40960. 59 

40240. 59 

0. 9824 

-1.  9995 

-8190.00 

8 

40960. 64 

40600. 64 

0. 9912 

-1. 1206 

-4589.99 

9 

40960.64 

41590.64 

1.0154 

1.  2964 

5309.99 

10 

40960. 59 

40960. 59 

1.  0000 

-. 2417 

-990.00 

MEAN 

1.  1645 

4770.  00 

MEAN 

AUDITED 

S.  D. 

DOLLARS  40960. 57 

0.05 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERR0R 

Numeric  Erro 

TOTA 

409605.  73 

410595. 73 

.  0024 

-990.  00 

AUD] 

157900. 17 

158170.  17 

0017 

-.  071 

-289. 60 

AUD] 

158128.98 

158218.98 

0006 

-.  185 

-756.87 

AUD] 

158031.95 

159291.95 

.  0080 

0.  556 

2275.82 

AUD] 

157685.03 

157595.03 

( 

9994 

-.299 

-1223. 79 

AUD] 

158113.10 

158563. 10 

0028 

0.043 

175.  76 

AUD] 

157829. 67 

157559. 67 

( 

9983 

-.  413 

-1690. 71 

AUD] 

157795.92 

158155.92 

0023 

-.014 

-55. 51 

AUD] 

157782.24 

158232.24 

0029 

0.  044 

178.21 

AUD] 

157933. 50 

158833. 50 

.  0057 

0.  328 

1344. 18 

AUD] 

158021.11 

158291. 11 

.0017 

-.  071 

-290.  13 

MEAN  PRED 

S.  D. 

MEAN  %  ERR 

MEAN  ERROR 

410562. 47 


DOLLARS 

CONTRACTS 


1161.85 

MEAN  AUDITED 
157922.  17 
830.90 


-.008 


■33. 265 


S.  D. 
149.81 
0.32 
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POPULATION  DD 


BASKET 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERR0R 

Numeric  1 

TOTAL 

372255.73 

372687. 17 

1.0012 

-431.  44 

1 

37225.  51 

37252. 65 

1.0007 

-00.04 

-160. 04 

2 

37225. 53 

37351. 67 

1.0034 

000.22 

829.96 

3 

37225. 56 

37265. 96 

1.0011 

-00.01 

-27. 44 

4 

37225. 56 

37097. 64 

0. 9966 

-00. 46 

-1710. 64 

5 

37225.57 

37253.81 

1.0008 

-00.04 

-149.04 

6 

37225.61 

37245. 16 

1.0005 

-00.06 

-235.94 

7 

37225.61 

37347. 20 

1.0033 

000. 21 

784.46 

8 

37225.61 

37272. 67 

1.  0013 

000.01 

39.  16 

9 

37225.61 

37382. 72 

1.  0042 

000.  31 

1139.66 

10 

37225. 56 

37217. 69 

0.  9998 

-00.  14 

-510. 14 

MEAN 

000. 15 

558.65 

MEAN 

AUDITED 

S.  D. 

DOLLARS  37225. 57 

0.04 

CONTRACTS 

830. 00 

0.  00 

DUS 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERR0R 

Numeric  1 

TOTAL 

372255.73 

372687. 17 

1.  0012 

-431. 44 

aud: 

IT 

153449.  50 

153530. 55 

1.  0005 

-.063 

-234.82 

aud: 

IT 

153678.01 

153708.72 

1.  0002 

-.096 

-357.05 

aud: 

IT 

153419.97 

153541.36 

1.  0008 

-.037 

-136.90 

aud: 

IT 

153377. 53 

153443.02 

1. 0004 

-.073 

-272.49 

aud: 

IT 

153713. 33 

153672.20 

0.  9997 

-.  143 

-531.  05 

aud: 

IT 

153722.20 

153766.66 

1.  0003 

-.  087 

-323.  78 

aud: 

IT 

153719.  79 

153736. 26 

1.  0001 

-.  105 

-391.  56 

aud: 

IT 

153579.34 

153665.42 

1.  0006 

-.  060 

-222. 79 

aud: 

IT 

153489.44 

153547.29 

1.0004 

-.078 

-291. 14 

aud: 

IT 

153472.87 

153584. 51 

1.  0007 

-.  043 

-160.  65 

m 

EAN 

PRED 

S.  D. 

MEAN  %  ERR 

MEAN  ER! 

372394.95 

329.31 

-.079 

-292.  2, 

DOLLARS 

CONTRACTS 


MEAN  AUDITED 
153562. 20 
831.30 


S.  D. 
136. 34 
0.  48 
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POPULATION  EE 


BASKET 

TOTAL 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

MEAN 


BOOK  VALUE 
413614. 33 
41361.43 
41361. 45 
41361. 39 
41361. 39 

41361. 39 
41361.37 

41361. 40 
41361. 50 

41361. 50 

41361. 51 


AUDIT  VALUE 
416811. 13 
41061. 73 

40861.95 
41910.84 
42160. 59 
41361. 39 
41661.07 
41960.80 

40961.90 

41910.95 

42959.91 


DOLLARS 

CONTRACTS 


MEAN  AUDITED 
41361. 43 
830. 00 


DUS 

TOTAL 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 

AUDIT 


MEAN  PRED 
415643.09 


DOLLARS 

CONTRACTS 


BOOK  VALUE 

413614. 33 

155935. 54 
155671. 11 

155685. 54 
155821. 71 
155915. 56 
155840.  41 
155740.43 
155955. 47 

155894. 34 
155769.62 


AUDIT  VALUE 
416811. 13 
156584.89 
156320.46 
156234.99 

156321. 21 
156864. 61 
156339.91 
157089.08 
156005.42 
156943. 29 

157168.22 
S.  D. 

1659.85 


MEAN  AUDITED 
155822.97 
830. 40 


FACTOR 

%ERROR 

Numeric  Error 

1. 0077 

-3196.80 

0. 9928 

-1. 4975 

-6193.80 

0. 9879 

-1.  9805 

-8191.  80 

1.  0133 

0. 5555 

2297. 71 

1.  0193 

1. 1593 

4795. 21 

1.  0000 

-.  7729 

-3196.80 

1.  0072 

-. 0483 

-199.80 

1.  0145 

0. 6763 

2797. 20 

0. 9903 

-1.  7390 

-7192. 79 

1.0133 

0. 5555 

2297.69 

1.  0386 

3. 0916 

12787. 17 

1.  2076 

4995. 00 

S.  D. 

0.05 

0.  00 

FACTOR 

%ERROR 

Numeric  Error 

1.  0077 

-3196.80 

1.  0042 

-.  356 

-1474. 42 

1.0042 

-.  356 

-1471.49 

1.  0035 

-.  420 

-1737. 06 

1.  0032 

-.  452 

-1870. 92 

1.  0061 

-.  164 

-679. 15 

1.  0032 

-.452 

-1871. 08 

1.0087 

0.  093 

384.  94 

1.  0003 

-.741 

-3064. 33 

1.  0067 

-.  100 

-413. 76 

1.  0090 

0.  125 

516. 90 

MEAN  %  ERR  MEAN  ERROR 
-.282  -1168.038 


S.  D. 
103. 18 
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